<!DOCTYPE html>
<html class="client-nojs vector-feature-night-mode-disabled vector-feature-language-in-header-enabled vector-feature-language-in-main-page-header-disabled vector-feature-page-tools-pinned-disabled vector-feature-toc-pinned-clientpref-1 vector-feature-main-menu-pinned-disabled vector-feature-limited-width-clientpref-1 vector-feature-limited-width-content-enabled vector-feature-custom-font-size-clientpref-1 vector-feature-appearance-pinned-clientpref-1 vector-sticky-header-enabled" lang="en" dir="ltr"><head>
<meta charset="UTF-8">
<title>Sparse dictionary learning</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="canonical" href="https://en.wikipedia.org/wiki/Sparse_dictionary_learning"> <link href="./mw/ext.cite.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/ext.math.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/skins.vector.icons.css" rel="stylesheet" type="text/css">
<link href="./mw/skins.vector.search.codex.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/skins.vector.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/user.styles.css" rel="stylesheet" type="text/css">
<meta name="ResourceLoaderDynamicStyles" content="">
<link rel="stylesheet" type="text/css" href="./mw/site.styles.css">
<link rel="stylesheet" type="text/css" href="./mw/noscript.css">
<link rel="stylesheet" type="text/css" href="./footer.css">
<link rel="stylesheet" type="text/css" href="./vector-2022.css">
</head>
<body class="skin--responsive skin-vector skin-vector-search-vue mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject page-Sparse_dictionary_learning rootpage-Sparse_dictionary_learning skin-vector-2022 action-view">
<div class="mw-page-container">
<div class="mw-page-container-inner">
<div class="mw-content-container">
<main id="content" class="mw-body">
<header class="mw-body-header vector-page-titlebar">
<h1 id="firstHeading" class="firstHeading mw-first-heading">
<span id="openzim-page-title" class="mw-page-title-main"><span class="mw-page-title-main">Sparse dictionary learning</span></span>
</h1>
</header>
<a id="top"></a>
<div id="bodyContent" class="vector-body ve-init-mw-desktopArticleTarget-targetContainer" aria-labelledby="firstHeading" data-mw-ve-target-container="">
<div id="mw-content-text" class="mw-body-content mw-content-ltr" lang="en" dir="ltr"><div class="mw-content-ltr mw-parser-output" lang="en" dir="ltr">
<style data-mw-deduplicate="TemplateStyles:r1129693374">
/* start https://en.wikipedia.org/ */
.mw-parser-output .hlist dl,.mw-parser-output .hlist ol,.mw-parser-output .hlist ul{margin:0;padding:0}.mw-parser-output .hlist dd,.mw-parser-output .hlist dt,.mw-parser-output .hlist li{margin:0;display:inline}.mw-parser-output .hlist.inline,.mw-parser-output .hlist.inline dl,.mw-parser-output .hlist.inline ol,.mw-parser-output .hlist.inline ul,.mw-parser-output .hlist dl dl,.mw-parser-output .hlist dl ol,.mw-parser-output .hlist dl ul,.mw-parser-output .hlist ol dl,.mw-parser-output .hlist ol ol,.mw-parser-output .hlist ol ul,.mw-parser-output .hlist ul dl,.mw-parser-output .hlist ul ol,.mw-parser-output .hlist ul ul{display:inline}.mw-parser-output .hlist .mw-empty-li{display:none}.mw-parser-output .hlist dt::after{content:": "}.mw-parser-output .hlist dd::after,.mw-parser-output .hlist li::after{content:" · ";font-weight:bold}.mw-parser-output .hlist dd:last-child::after,.mw-parser-output .hlist dt:last-child::after,.mw-parser-output .hlist li:last-child::after{content:none}.mw-parser-output .hlist dd dd:first-child::before,.mw-parser-output .hlist dd dt:first-child::before,.mw-parser-output .hlist dd li:first-child::before,.mw-parser-output .hlist dt dd:first-child::before,.mw-parser-output .hlist dt dt:first-child::before,.mw-parser-output .hlist dt li:first-child::before,.mw-parser-output .hlist li dd:first-child::before,.mw-parser-output .hlist li dt:first-child::before,.mw-parser-output .hlist li li:first-child::before{content:" (";font-weight:normal}.mw-parser-output .hlist dd dd:last-child::after,.mw-parser-output .hlist dd dt:last-child::after,.mw-parser-output .hlist dd li:last-child::after,.mw-parser-output .hlist dt dd:last-child::after,.mw-parser-output .hlist dt dt:last-child::after,.mw-parser-output .hlist dt li:last-child::after,.mw-parser-output .hlist li dd:last-child::after,.mw-parser-output .hlist li dt:last-child::after,.mw-parser-output .hlist li li:last-child::after{content:")";font-weight:normal}.mw-parser-output .hlist ol{counter-reset:listitem}.mw-parser-output .hlist ol>li{counter-increment:listitem}.mw-parser-output .hlist ol>li::before{content:" "counter(listitem)"\a0 "}.mw-parser-output .hlist dd ol>li:first-child::before,.mw-parser-output .hlist dt ol>li:first-child::before,.mw-parser-output .hlist li ol>li:first-child::before{content:" ("counter(listitem)"\a0 "}
/* end https://en.wikipedia.org/ */
</style><style data-mw-deduplicate="TemplateStyles:r1246091330">
/* start https://en.wikipedia.org/ */
.mw-parser-output .sidebar{width:22em;float:right;clear:right;margin:0.5em 0 1em 1em;background:var(--background-color-neutral-subtle,#f8f9fa);border:1px solid var(--border-color-base,#a2a9b1);padding:0.2em;text-align:center;line-height:1.4em;font-size:88%;border-collapse:collapse;display:table}body.skin-minerva .mw-parser-output .sidebar{display:table!important;float:right!important;margin:0.5em 0 1em 1em!important}.mw-parser-output .sidebar-subgroup{width:100%;margin:0;border-spacing:0}.mw-parser-output .sidebar-left{float:left;clear:left;margin:0.5em 1em 1em 0}.mw-parser-output .sidebar-none{float:none;clear:both;margin:0.5em 1em 1em 0}.mw-parser-output .sidebar-outer-title{padding:0 0.4em 0.2em;font-size:125%;line-height:1.2em;font-weight:bold}.mw-parser-output .sidebar-top-image{padding:0.4em}.mw-parser-output .sidebar-top-caption,.mw-parser-output .sidebar-pretitle-with-top-image,.mw-parser-output .sidebar-caption{padding:0.2em 0.4em 0;line-height:1.2em}.mw-parser-output .sidebar-pretitle{padding:0.4em 0.4em 0;line-height:1.2em}.mw-parser-output .sidebar-title,.mw-parser-output .sidebar-title-with-pretitle{padding:0.2em 0.8em;font-size:145%;line-height:1.2em}.mw-parser-output .sidebar-title-with-pretitle{padding:0.1em 0.4em}.mw-parser-output .sidebar-image{padding:0.2em 0.4em 0.4em}.mw-parser-output .sidebar-heading{padding:0.1em 0.4em}.mw-parser-output .sidebar-content{padding:0 0.5em 0.4em}.mw-parser-output .sidebar-content-with-subgroup{padding:0.1em 0.4em 0.2em}.mw-parser-output .sidebar-above,.mw-parser-output .sidebar-below{padding:0.3em 0.8em;font-weight:bold}.mw-parser-output .sidebar-collapse .sidebar-above,.mw-parser-output .sidebar-collapse .sidebar-below{border-top:1px solid #aaa;border-bottom:1px solid #aaa}.mw-parser-output .sidebar-navbar{text-align:right;font-size:115%;padding:0 0.4em 0.4em}.mw-parser-output .sidebar-list-title{padding:0 0.4em;text-align:left;font-weight:bold;line-height:1.6em;font-size:105%}.mw-parser-output .sidebar-list-title-c{padding:0 0.4em;text-align:center;margin:0 3.3em}@media(max-width:640px){body.mediawiki .mw-parser-output .sidebar{width:100%!important;clear:both;float:none!important;margin-left:0!important;margin-right:0!important}}body.skin--responsive .mw-parser-output .sidebar a>img{max-width:none!important}@media screen{html.skin-theme-clientpref-night .mw-parser-output .sidebar:not(.notheme) .sidebar-list-title,html.skin-theme-clientpref-night .mw-parser-output .sidebar:not(.notheme) .sidebar-title-with-pretitle{background:transparent!important}html.skin-theme-clientpref-night .mw-parser-output .sidebar:not(.notheme) .sidebar-title-with-pretitle a{color:var(--color-progressive)!important}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .sidebar:not(.notheme) .sidebar-list-title,html.skin-theme-clientpref-os .mw-parser-output .sidebar:not(.notheme) .sidebar-title-with-pretitle{background:transparent!important}html.skin-theme-clientpref-os .mw-parser-output .sidebar:not(.notheme) .sidebar-title-with-pretitle a{color:var(--color-progressive)!important}}@media print{body.ns-0 .mw-parser-output .sidebar{display:none!important}}
/* end https://en.wikipedia.org/ */
</style><style data-mw-deduplicate="TemplateStyles:r886047488">
/* start https://en.wikipedia.org/ */
.mw-parser-output .nobold{font-weight:normal}
/* end https://en.wikipedia.org/ */
</style><table class="sidebar sidebar-collapse nomobile nowraplinks"><tbody><tr><td class="sidebar-pretitle">Part of a series on</td></tr><tr><th class="sidebar-title-with-pretitle"><a href="Machine_learning" title="Machine learning">Machine learning</a><br>and <a href="Data_mining" title="Data mining">data mining</a></th></tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Paradigms</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Supervised_learning" title="Supervised learning">Supervised learning</a></li>
<li><a href="Unsupervised_learning" title="Unsupervised learning">Unsupervised learning</a></li>
<li><a href="Semi-supervised_learning" class="mw-redirect" title="Semi-supervised learning">Semi-supervised learning</a></li>
<li><a href="Self-supervised_learning" title="Self-supervised learning">Self-supervised learning</a></li>
<li><a href="Reinforcement_learning" title="Reinforcement learning">Reinforcement learning</a></li>
<li><a href="Meta-learning_(computer_science)" title="Meta-learning (computer science)">Meta-learning</a></li>
<li><a href="Online_machine_learning" title="Online machine learning">Online learning</a></li>
<li><a href="Batch_learning" class="mw-redirect" title="Batch learning">Batch learning</a></li>
<li><a href="Curriculum_learning" title="Curriculum learning">Curriculum learning</a></li>
<li><a href="Rule-based_machine_learning" title="Rule-based machine learning">Rule-based learning</a></li>
<li><a href="Neuro-symbolic_AI" title="Neuro-symbolic AI">Neuro-symbolic AI</a></li>
<li><a href="Neuromorphic_engineering" class="mw-redirect" title="Neuromorphic engineering">Neuromorphic engineering</a></li>
<li><a href="Quantum_machine_learning" title="Quantum machine learning">Quantum machine learning</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Problems</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Statistical_classification" title="Statistical classification">Classification</a></li>
<li><a href="Generative_model" title="Generative model">Generative modeling</a></li>
<li><a href="Regression_analysis" title="Regression analysis">Regression</a></li>
<li><a href="Cluster_analysis" title="Cluster analysis">Clustering</a></li>
<li><a href="Dimensionality_reduction" title="Dimensionality reduction">Dimensionality reduction</a></li>
<li><a href="Density_estimation" title="Density estimation">Density estimation</a></li>
<li><a href="Anomaly_detection" title="Anomaly detection">Anomaly detection</a></li>
<li><a href="Data_cleaning" class="mw-redirect" title="Data cleaning">Data cleaning</a></li>
<li><a href="Automated_machine_learning" title="Automated machine learning">AutoML</a></li>
<li><a href="Association_rule_learning" title="Association rule learning">Association rules</a></li>
<li><a href="Semantic_analysis_(machine_learning)" title="Semantic analysis (machine learning)">Semantic analysis</a></li>
<li><a href="Structured_prediction" title="Structured prediction">Structured prediction</a></li>
<li><a href="Feature_engineering" title="Feature engineering">Feature engineering</a></li>
<li><a href="Feature_learning" title="Feature learning">Feature learning</a></li>
<li><a href="Learning_to_rank" title="Learning to rank">Learning to rank</a></li>
<li><a href="Grammar_induction" title="Grammar induction">Grammar induction</a></li>
<li><a href="Ontology_learning" title="Ontology learning">Ontology learning</a></li>
<li><a href="Multimodal_learning" title="Multimodal learning">Multimodal learning</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><div style="display: inline-block; line-height: 1.2em; padding: .1em 0;"><a href="Supervised_learning" title="Supervised learning">Supervised learning</a><br><span class="nobold"><span style="font-size: 85%;">(<b><a href="Statistical_classification" title="Statistical classification">classification</a></b> • <b><a href="Regression_analysis" title="Regression analysis">regression</a></b>)</span></span> </div></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Apprenticeship_learning" title="Apprenticeship learning">Apprenticeship learning</a></li>
<li><a href="Decision_tree_learning" title="Decision tree learning">Decision trees</a></li>
<li><a href="Ensemble_learning" title="Ensemble learning">Ensembles</a>
<ul><li><a href="Bootstrap_aggregating" title="Bootstrap aggregating">Bagging</a></li>
<li><a href="Boosting_(machine_learning)" title="Boosting (machine learning)">Boosting</a></li>
<li><a href="Random_forest" title="Random forest">Random forest</a></li></ul></li>
<li><a href="K-nearest_neighbors_algorithm" title="K-nearest neighbors algorithm"><i>k</i>-NN</a></li>
<li><a href="Linear_regression" title="Linear regression">Linear regression</a></li>
<li><a href="Naive_Bayes_classifier" title="Naive Bayes classifier">Naive Bayes</a></li>
<li><a href="Artificial_neural_network" class="mw-redirect" title="Artificial neural network">Artificial neural networks</a></li>
<li><a href="Logistic_regression" title="Logistic regression">Logistic regression</a></li>
<li><a href="Perceptron" title="Perceptron">Perceptron</a></li>
<li><a href="Relevance_vector_machine" title="Relevance vector machine">Relevance vector machine (RVM)</a></li>
<li><a href="Support_vector_machine" title="Support vector machine">Support vector machine (SVM)</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Cluster_analysis" title="Cluster analysis">Clustering</a></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="BIRCH" title="BIRCH">BIRCH</a></li>
<li><a href="CURE_algorithm" title="CURE algorithm">CURE</a></li>
<li><a href="Hierarchical_clustering" title="Hierarchical clustering">Hierarchical</a></li>
<li><a href="K-means_clustering" title="K-means clustering"><i>k</i>-means</a></li>
<li><a href="Fuzzy_clustering" title="Fuzzy clustering">Fuzzy</a></li>
<li><a href="Expectation%E2%80%93maximization_algorithm" title="Expectation–maximization algorithm">Expectation–maximization (EM)</a></li>
<li><br><a href="DBSCAN" title="DBSCAN">DBSCAN</a></li>
<li><a href="OPTICS_algorithm" title="OPTICS algorithm">OPTICS</a></li>
<li><a href="Mean_shift" title="Mean shift">Mean shift</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Dimensionality_reduction" title="Dimensionality reduction">Dimensionality reduction</a></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Factor_analysis" title="Factor analysis">Factor analysis</a></li>
<li><a href="Canonical_correlation" title="Canonical correlation">CCA</a></li>
<li><a href="Independent_component_analysis" title="Independent component analysis">ICA</a></li>
<li><a href="Linear_discriminant_analysis" title="Linear discriminant analysis">LDA</a></li>
<li><a href="Non-negative_matrix_factorization" title="Non-negative matrix factorization">NMF</a></li>
<li><a href="Principal_component_analysis" title="Principal component analysis">PCA</a></li>
<li><a href="Proper_generalized_decomposition" title="Proper generalized decomposition">PGD</a></li>
<li><a href="T-distributed_stochastic_neighbor_embedding" title="T-distributed stochastic neighbor embedding">t-SNE</a></li>
</ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Structured_prediction" title="Structured prediction">Structured prediction</a></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Graphical_model" title="Graphical model">Graphical models</a>
<ul><li><a href="Bayesian_network" title="Bayesian network">Bayes net</a></li>
<li><a href="Conditional_random_field" title="Conditional random field">Conditional random field</a></li>
<li><a href="Hidden_Markov_model" title="Hidden Markov model">Hidden Markov</a></li></ul></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Anomaly_detection" title="Anomaly detection">Anomaly detection</a></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Random_sample_consensus" title="Random sample consensus">RANSAC</a></li>
<li><a href="K-nearest_neighbors_algorithm" title="K-nearest neighbors algorithm"><i>k</i>-NN</a></li>
<li><a href="Local_outlier_factor" title="Local outlier factor">Local outlier factor</a></li>
<li><a href="Isolation_forest" title="Isolation forest">Isolation forest</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Neural_network_(machine_learning)" title="Neural network (machine learning)">Neural networks</a></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Autoencoder" title="Autoencoder">Autoencoder</a></li>
<li><a href="Deep_learning" title="Deep learning">Deep learning</a></li>
<li><a href="Feedforward_neural_network" title="Feedforward neural network">Feedforward neural network</a></li>
<li><a href="Recurrent_neural_network" title="Recurrent neural network">Recurrent neural network</a>
<ul><li><a href="Long_short-term_memory" title="Long short-term memory">LSTM</a></li>
<li><a href="Gated_recurrent_unit" title="Gated recurrent unit">GRU</a></li>
<li><a href="Echo_state_network" title="Echo state network">ESN</a></li>
<li><a href="Reservoir_computing" title="Reservoir computing">reservoir computing</a></li></ul></li>
<li><a href="Boltzmann_machine" title="Boltzmann machine">Boltzmann machine</a>
<ul><li><a href="Restricted_Boltzmann_machine" title="Restricted Boltzmann machine">Restricted</a></li></ul></li>
<li><a href="Generative_adversarial_network" title="Generative adversarial network">GAN</a></li>
<li><a href="Diffusion_model" title="Diffusion model">Diffusion model</a></li>
<li><a href="Self-organizing_map" title="Self-organizing map">SOM</a></li>
<li><a href="Convolutional_neural_network" title="Convolutional neural network">Convolutional neural network</a>
<ul><li><a href="U-Net" title="U-Net">U-Net</a></li>
<li><a href="LeNet" title="LeNet">LeNet</a></li>
<li><a href="AlexNet" title="AlexNet">AlexNet</a></li>
<li><a href="DeepDream" title="DeepDream">DeepDream</a></li></ul></li>
<li><a href="Neural_field" title="Neural field">Neural field</a>
<ul><li><a href="Neural_radiance_field" title="Neural radiance field">Neural radiance field</a></li>
<li><a href="Physics-informed_neural_networks" title="Physics-informed neural networks">Physics-informed neural networks</a></li></ul></li>
<li><a href="Transformer_(deep_learning_architecture)" title="Transformer (deep learning architecture)">Transformer</a>
<ul><li><a href="Vision_transformer" title="Vision transformer">Vision</a></li></ul></li>
<li><a href="Mamba_(deep_learning_architecture)" title="Mamba (deep learning architecture)">Mamba</a></li>
<li><a href="Spiking_neural_network" title="Spiking neural network">Spiking neural network</a></li>
<li><a href="Memtransistor" title="Memtransistor">Memtransistor</a></li>
<li><a href="Electrochemical_RAM" title="Electrochemical RAM">Electrochemical RAM</a> (ECRAM)</li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Reinforcement_learning" title="Reinforcement learning">Reinforcement learning</a></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Q-learning" title="Q-learning">Q-learning</a></li>
<li><a href="Policy_gradient_method" title="Policy gradient method">Policy gradient</a></li>
<li><a href="State%E2%80%93action%E2%80%93reward%E2%80%93state%E2%80%93action" title="State–action–reward–state–action">SARSA</a></li>
<li><a href="Temporal_difference_learning" title="Temporal difference learning">Temporal difference (TD)</a></li>
<li><a href="Multi-agent_reinforcement_learning" title="Multi-agent reinforcement learning">Multi-agent</a>
<ul><li><a href="Self-play_(reinforcement_learning_technique)" class="mw-redirect" title="Self-play (reinforcement learning technique)">Self-play</a></li></ul></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Learning with humans</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Active_learning_(machine_learning)" title="Active learning (machine learning)">Active learning</a></li>
<li><a href="Crowdsourcing" title="Crowdsourcing">Crowdsourcing</a></li>
<li><a href="Human-in-the-loop" title="Human-in-the-loop">Human-in-the-loop</a></li>
<li><a href="Mechanistic_interpretability" title="Mechanistic interpretability">Mechanistic interpretability</a></li>
<li><a href="Reinforcement_learning_from_human_feedback" title="Reinforcement learning from human feedback">RLHF</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Model diagnostics</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Coefficient_of_determination" title="Coefficient of determination">Coefficient of determination</a></li>
<li><a href="Confusion_matrix" title="Confusion matrix">Confusion matrix</a></li>
<li><a href="Learning_curve_(machine_learning)" title="Learning curve (machine learning)">Learning curve</a></li>
<li><a href="Receiver_operating_characteristic" title="Receiver operating characteristic">ROC curve</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Mathematical foundations</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Kernel_machines" class="mw-redirect" title="Kernel machines">Kernel machines</a></li>
<li><a href="Bias%E2%80%93variance_tradeoff" title="Bias–variance tradeoff">Bias–variance tradeoff</a></li>
<li><a href="Computational_learning_theory" title="Computational learning theory">Computational learning theory</a></li>
<li><a href="Empirical_risk_minimization" title="Empirical risk minimization">Empirical risk minimization</a></li>
<li><a href="Occam_learning" title="Occam learning">Occam learning</a></li>
<li><a href="Probably_approximately_correct_learning" title="Probably approximately correct learning">PAC learning</a></li>
<li><a href="Statistical_learning_theory" title="Statistical learning theory">Statistical learning</a></li>
<li><a href="Vapnik%E2%80%93Chervonenkis_theory" title="Vapnik–Chervonenkis theory">VC theory</a></li>
<li><a href="Topological_deep_learning" title="Topological deep learning">Topological deep learning</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Journals and conferences</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="AAAI_Conference_on_Artificial_Intelligence" title="AAAI Conference on Artificial Intelligence">AAAI</a></li>
<li><a href="ECML_PKDD" title="ECML PKDD">ECML PKDD</a></li>
<li><a href="Conference_on_Neural_Information_Processing_Systems" title="Conference on Neural Information Processing Systems">NeurIPS</a></li>
<li><a href="International_Conference_on_Machine_Learning" title="International Conference on Machine Learning">ICML</a></li>
<li><a href="International_Conference_on_Learning_Representations" title="International Conference on Learning Representations">ICLR</a></li>
<li><a href="International_Joint_Conference_on_Artificial_Intelligence" title="International Joint Conference on Artificial Intelligence">IJCAI</a></li>
<li><a href="Machine_Learning_(journal)" title="Machine Learning (journal)">ML</a></li>
<li><a href="Journal_of_Machine_Learning_Research" title="Journal of Machine Learning Research">JMLR</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Related articles</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Glossary_of_artificial_intelligence" title="Glossary of artificial intelligence">Glossary of artificial intelligence</a></li>
<li><a href="List_of_datasets_for_machine-learning_research" title="List of datasets for machine-learning research">List of datasets for machine-learning research</a>
<ul><li><a href="List_of_datasets_in_computer_vision_and_image_processing" title="List of datasets in computer vision and image processing">List of datasets in computer vision and image processing</a></li></ul></li>
<li><a href="Outline_of_machine_learning" title="Outline of machine learning">Outline of machine learning</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-navbar"><style data-mw-deduplicate="TemplateStyles:r1239400231">
/* start https://en.wikipedia.org/ */
.mw-parser-output .navbar{display:inline;font-size:88%;font-weight:normal}.mw-parser-output .navbar-collapse{float:left;text-align:left}.mw-parser-output .navbar-boxtext{word-spacing:0}.mw-parser-output .navbar ul{display:inline-block;white-space:nowrap;line-height:inherit}.mw-parser-output .navbar-brackets::before{margin-right:-0.125em;content:"[ "}.mw-parser-output .navbar-brackets::after{margin-left:-0.125em;content:" ]"}.mw-parser-output .navbar li{word-spacing:-0.125em}.mw-parser-output .navbar a>span,.mw-parser-output .navbar a>abbr{text-decoration:inherit}.mw-parser-output .navbar-mini abbr{font-variant:small-caps;border-bottom:none;text-decoration:none;cursor:inherit}.mw-parser-output .navbar-ct-full{font-size:114%;margin:0 7em}.mw-parser-output .navbar-ct-mini{font-size:114%;margin:0 4em}html.skin-theme-clientpref-night .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}@media(prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}}@media print{.mw-parser-output .navbar{display:none!important}}
/* end https://en.wikipedia.org/ */
</style></td></tr></tbody></table><p><b>Sparse dictionary learning</b> (also known as <b>sparse coding</b> or <b>SDL</b>) is a <a href="Representation_learning" class="mw-redirect" title="Representation learning">representation learning</a> method which aims to find a <a href="Sparse_matrix" title="Sparse matrix">sparse</a> representation of the input data in the form of a <a href="Linear_combination" title="Linear combination">linear combination</a> of basic elements as well as those basic elements themselves. These elements are called <i>atoms</i>, and they compose a <i>dictionary</i>. Atoms in the dictionary are not required to be <a href="Orthogonal_basis" title="Orthogonal basis">orthogonal</a>, and they may be an over-complete spanning set. This problem setup also allows the dimensionality of the signals being represented to be higher than any one of the signals being observed. These two properties lead to having seemingly redundant atoms that allow multiple representations of the same signal, but also provide an improvement in <a href="Sparsity" class="mw-redirect" title="Sparsity">sparsity</a> and flexibility of the representation.
</p><p>One of the most important applications of sparse dictionary learning is in the field of <a href="Compressed_sensing" title="Compressed sensing">compressed sensing</a> or <a href="Detection_theory" title="Detection theory">signal recovery</a>. In compressed sensing, a high-dimensional signal can be recovered with only a few linear measurements, provided that the signal is sparse or near-sparse. Since not all signals satisfy this condition, it is crucial to find a sparse representation of that signal such as the <a href="Wavelet_transform" title="Wavelet transform">wavelet transform</a> or the directional gradient of a rasterized matrix. Once a matrix or a high-dimensional vector is transferred to a sparse space, different recovery algorithms like <a href="Basis_pursuit" title="Basis pursuit">basis pursuit</a>, CoSaMP,<sup id="cite_ref-1" class="reference"><a href="#cite_note-1"><span class="cite-bracket">[</span>1<span class="cite-bracket">]</span></a></sup> or fast non-iterative algorithms<sup id="cite_ref-2" class="reference"><a href="#cite_note-2"><span class="cite-bracket">[</span>2<span class="cite-bracket">]</span></a></sup> can be used to recover the signal.
</p><p>One of the key principles of dictionary learning is that the dictionary has to be inferred from the input data. The emergence of sparse dictionary learning methods was stimulated by the fact that in <a href="Signal_processing" title="Signal processing">signal processing</a>, one typically wants to represent the input data using a minimal amount of components. Before this approach, the general practice was to use predefined dictionaries such as <a href="Fourier_transform" title="Fourier transform">Fourier</a> or <a href="Wavelet_transform" title="Wavelet transform">wavelet</a> transforms. However, in certain cases, a dictionary that is trained to fit the input data can significantly improve the sparsity, which has applications in data decomposition, <a href="Data_compression" title="Data compression">compression</a>, and <a href="Data_analysis" title="Data analysis">analysis</a>, and has been used in the fields of image <a href="Noise_reduction" title="Noise reduction">denoising</a> and <a href="Image_classification" class="mw-redirect" title="Image classification">classification</a>, and video and <a href="Audio_signal_processing" title="Audio signal processing">audio processing</a>. Sparsity and overcomplete dictionaries have immense applications in image compression, image fusion, and <a href="Inpainting" title="Inpainting">inpainting</a>.
</p>
<meta property="mw:PageProp/toc">
<div class="mw-heading mw-heading2"><h2 id="Problem_statement">Problem statement</h2></div>
<p>Given the input dataset <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X=[x_{1},...,x_{K}],x_{i}\in \mathbb {R} ^{d}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>X</mi>
<mo>=</mo>
<mo stretchy="false">[</mo>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
</msub>
<mo>,</mo>
<mo>.</mo>
<mo>.</mo>
<mo>.</mo>
<mo>,</mo>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>K</mi>
</mrow>
</msub>
<mo stretchy="false">]</mo>
<mo>,</mo>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<mo>∈<!-- ∈ --></mo>
<msup>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="double-struck">R</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>d</mi>
</mrow>
</msup>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle X=[x_{1},...,x_{K}],x_{i}\in \mathbb {R} ^{d}}</annotation>
</semantics>
</math></span><img src="./f81aab97e24cae9838f71d752815ccd74c23ce9a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:25.723ex; height:3.176ex;" alt="{\displaystyle X=[x_{1},...,x_{K}],x_{i}\in \mathbb {R} ^{d}}" loading="lazy"></span> we wish to find a dictionary <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} \in \mathbb {R} ^{d\times n}:D=[d_{1},...,d_{n}]}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mo>∈<!-- ∈ --></mo>
<msup>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="double-struck">R</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>d</mi>
<mo>×<!-- × --></mo>
<mi>n</mi>
</mrow>
</msup>
<mo>:</mo>
<mi>D</mi>
<mo>=</mo>
<mo stretchy="false">[</mo>
<msub>
<mi>d</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
</msub>
<mo>,</mo>
<mo>.</mo>
<mo>.</mo>
<mo>.</mo>
<mo>,</mo>
<msub>
<mi>d</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>n</mi>
</mrow>
</msub>
<mo stretchy="false">]</mo>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} \in \mathbb {R} ^{d\times n}:D=[d_{1},...,d_{n}]}</annotation>
</semantics>
</math></span><img src="./8737bebe3150b98f667b894ca4c795c2d49d734d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:28.039ex; height:3.176ex;" alt="{\displaystyle \mathbf {D} \in \mathbb {R} ^{d\times n}:D=[d_{1},...,d_{n}]}" loading="lazy"></span> and a representation <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle R=[r_{1},...,r_{K}],r_{i}\in \mathbb {R} ^{n}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>R</mi>
<mo>=</mo>
<mo stretchy="false">[</mo>
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
</msub>
<mo>,</mo>
<mo>.</mo>
<mo>.</mo>
<mo>.</mo>
<mo>,</mo>
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>K</mi>
</mrow>
</msub>
<mo stretchy="false">]</mo>
<mo>,</mo>
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<mo>∈<!-- ∈ --></mo>
<msup>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="double-struck">R</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>n</mi>
</mrow>
</msup>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle R=[r_{1},...,r_{K}],r_{i}\in \mathbb {R} ^{n}}</annotation>
</semantics>
</math></span><img src="./33e741160490ff5f71f9712d3800e9f502fbbb71.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:24.79ex; height:2.843ex;" alt="{\displaystyle R=[r_{1},...,r_{K}],r_{i}\in \mathbb {R} ^{n}}" loading="lazy"></span> such that both <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \|X-\mathbf {D} R\|_{F}^{2}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mi>X</mi>
<mo>−<!-- − --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mi>R</mi>
<msubsup>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>F</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \|X-\mathbf {D} R\|_{F}^{2}}</annotation>
</semantics>
</math></span><img src="./0e8cb1c56f280cf64283c98f0477add3dca91a59.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:12.422ex; height:3.176ex;" alt="{\displaystyle \|X-\mathbf {D} R\|_{F}^{2}}" loading="lazy"></span> is minimized and the representations <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle r_{i}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle r_{i}}</annotation>
</semantics>
</math></span><img src="./a0b6d651eaf432dbf1f106021c8bb499ae83fd1f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.848ex; height:2.009ex;" alt="{\displaystyle r_{i}}" loading="lazy"></span> are sparse enough. This can be formulated as the following <a href="Optimization_problem" title="Optimization problem">optimization problem</a>:
</p><p><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\underset {\mathbf {D} \in {\mathcal {C}},r_{i}\in \mathbb {R} ^{n}}{\text{argmin}}}\sum _{i=1}^{K}\|x_{i}-\mathbf {D} r_{i}\|_{2}^{2}+\lambda \|r_{i}\|_{0}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<munder>
<mtext>argmin</mtext>
<mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mo>∈<!-- ∈ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mrow class="MJX-TeXAtom-ORD">
<mi class="MJX-tex-caligraphic" mathvariant="script">C</mi>
</mrow>
</mrow>
<mo>,</mo>
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<mo>∈<!-- ∈ --></mo>
<msup>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="double-struck">R</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>n</mi>
</mrow>
</msup>
</mrow>
</munder>
</mrow>
<munderover>
<mo>∑<!-- ∑ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>K</mi>
</mrow>
</munderover>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<mo>−<!-- − --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<msubsup>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo>+</mo>
<mi>λ<!-- λ --></mi>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<msub>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mn>0</mn>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle {\underset {\mathbf {D} \in {\mathcal {C}},r_{i}\in \mathbb {R} ^{n}}{\text{argmin}}}\sum _{i=1}^{K}\|x_{i}-\mathbf {D} r_{i}\|_{2}^{2}+\lambda \|r_{i}\|_{0}}</annotation>
</semantics>
</math></span><img src="./81449a31e07ad388801379c804b73e6d1f044ce2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.005ex; width:34.292ex; height:7.343ex;" alt="{\displaystyle {\underset {\mathbf {D} \in {\mathcal {C}},r_{i}\in \mathbb {R} ^{n}}{\text{argmin}}}\sum _{i=1}^{K}\|x_{i}-\mathbf {D} r_{i}\|_{2}^{2}+\lambda \|r_{i}\|_{0}}" loading="lazy"></span>, where <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {C}}\equiv \{\mathbf {D} \in \mathbb {R} ^{d\times n}:\|d_{i}\|_{2}\leq 1\,\,\forall i=1,...,n\}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mrow class="MJX-TeXAtom-ORD">
<mi class="MJX-tex-caligraphic" mathvariant="script">C</mi>
</mrow>
</mrow>
<mo>≡<!-- ≡ --></mo>
<mo fence="false" stretchy="false">{</mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mo>∈<!-- ∈ --></mo>
<msup>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="double-struck">R</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>d</mi>
<mo>×<!-- × --></mo>
<mi>n</mi>
</mrow>
</msup>
<mo>:</mo>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<msub>
<mi>d</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<msub>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msub>
<mo>≤<!-- ≤ --></mo>
<mn>1</mn>
<mspace width="thinmathspace"></mspace>
<mspace width="thinmathspace"></mspace>
<mi mathvariant="normal">∀<!-- ∀ --></mi>
<mi>i</mi>
<mo>=</mo>
<mn>1</mn>
<mo>,</mo>
<mo>.</mo>
<mo>.</mo>
<mo>.</mo>
<mo>,</mo>
<mi>n</mi>
<mo fence="false" stretchy="false">}</mo>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle {\mathcal {C}}\equiv \{\mathbf {D} \in \mathbb {R} ^{d\times n}:\|d_{i}\|_{2}\leq 1\,\,\forall i=1,...,n\}}</annotation>
</semantics>
</math></span><img src="./8d58b68ba94043280df15d83b9244cbff40dfb69.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:41.868ex; height:3.176ex;" alt="{\displaystyle {\mathcal {C}}\equiv \{\mathbf {D} \in \mathbb {R} ^{d\times n}:\|d_{i}\|_{2}\leq 1\,\,\forall i=1,...,n\}}" loading="lazy"></span>, <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \lambda >0}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>λ<!-- λ --></mi>
<mo>></mo>
<mn>0</mn>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \lambda >0}</annotation>
</semantics>
</math></span><img src="./eea25afc0351140f919cf791c49c1964b8b081de.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:5.616ex; height:2.176ex;" alt="{\displaystyle \lambda >0}" loading="lazy"></span>
</p><p><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {C}}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mrow class="MJX-TeXAtom-ORD">
<mi class="MJX-tex-caligraphic" mathvariant="script">C</mi>
</mrow>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle {\mathcal {C}}}</annotation>
</semantics>
</math></span><img src="./e7b3edab7022ca9e2976651bc59c489513ee9019.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.239ex; height:2.176ex;" alt="{\displaystyle {\mathcal {C}}}" loading="lazy"></span> is required to constrain <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} }</annotation>
</semantics>
</math></span><img src="./b2345293072878db24e119c580def49ad582e3ed.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.05ex; height:2.176ex;" alt="{\displaystyle \mathbf {D} }" loading="lazy"></span> so that its atoms would not reach arbitrarily high values allowing for arbitrarily low (but non-zero) values of <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle r_{i}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle r_{i}}</annotation>
</semantics>
</math></span><img src="./a0b6d651eaf432dbf1f106021c8bb499ae83fd1f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.848ex; height:2.009ex;" alt="{\displaystyle r_{i}}" loading="lazy"></span>. <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \lambda }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>λ<!-- λ --></mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \lambda }</annotation>
</semantics>
</math></span><img src="./b43d0ea3c9c025af1be9128e62a18fa74bedda2a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.355ex; height:2.176ex;" alt="{\displaystyle \lambda }" loading="lazy"></span> controls the trade off between the sparsity and the minimization error.
</p><p>The minimization problem above is not convex because of the <a href="L0_norm" class="mw-redirect" title="L0 norm">ℓ<sub>0</sub>-"norm"</a> and solving this problem is NP-hard.<sup id="cite_ref-3" class="reference"><a href="#cite_note-3"><span class="cite-bracket">[</span>3<span class="cite-bracket">]</span></a></sup> In some cases <i><a href="L1-norm" class="mw-redirect" title="L1-norm">L</a></i><sup><a href="L1-norm" class="mw-redirect" title="L1-norm">1</a></sup><a href="L1-norm" class="mw-redirect" title="L1-norm">-norm</a> is known to ensure sparsity<sup id="cite_ref-4" class="reference"><a href="#cite_note-4"><span class="cite-bracket">[</span>4<span class="cite-bracket">]</span></a></sup> and so the above becomes a <a href="Convex_optimization" title="Convex optimization">convex optimization</a> problem with respect to each of the variables <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} }</annotation>
</semantics>
</math></span><img src="./b2345293072878db24e119c580def49ad582e3ed.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.05ex; height:2.176ex;" alt="{\displaystyle \mathbf {D} }" loading="lazy"></span> and <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {R} }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">R</mi>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {R} }</annotation>
</semantics>
</math></span><img src="./5de85fcc2a00d8ba14aae84aeef812d7fef4b3d5.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.003ex; height:2.176ex;" alt="{\displaystyle \mathbf {R} }" loading="lazy"></span> when the other one is fixed, but it is not jointly convex in <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (\mathbf {D} ,\mathbf {R} )}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mo stretchy="false">(</mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mo>,</mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">R</mi>
</mrow>
<mo stretchy="false">)</mo>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle (\mathbf {D} ,\mathbf {R} )}</annotation>
</semantics>
</math></span><img src="./448bbe0c050e6ad02efd294306fe356cb936d948.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:6.896ex; height:2.843ex;" alt="{\displaystyle (\mathbf {D} ,\mathbf {R} )}" loading="lazy"></span>.
</p>
<div class="mw-heading mw-heading3"><h3 id="Properties_of_the_dictionary">Properties of the dictionary</h3></div>
<p>The dictionary <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} }</annotation>
</semantics>
</math></span><img src="./b2345293072878db24e119c580def49ad582e3ed.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.05ex; height:2.176ex;" alt="{\displaystyle \mathbf {D} }" loading="lazy"></span> defined above can be "undercomplete" if <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n<d}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>n</mi>
<mo><</mo>
<mi>d</mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle n<d}</annotation>
</semantics>
</math></span><img src="./fb24d50a959320f2b673f848b6195d1a6ddf0dba.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:5.709ex; height:2.176ex;" alt="{\displaystyle n<d}" loading="lazy"></span> or "overcomplete" in case <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n>d}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>n</mi>
<mo>></mo>
<mi>d</mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle n>d}</annotation>
</semantics>
</math></span><img src="./dfe1896db4e754264895da715791d9bd9387b3ce.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:5.709ex; height:2.176ex;" alt="{\displaystyle n>d}" loading="lazy"></span> with the latter being a typical assumption for a sparse dictionary learning problem. The case of a complete dictionary does not provide any improvement from a representational point of view and thus isn't considered.
</p><p>Undercomplete dictionaries represent the setup in which the actual input data lies in a lower-dimensional space. This case is strongly related to <a href="Dimensionality_reduction" title="Dimensionality reduction">dimensionality reduction</a> and techniques like <a href="Principal_component_analysis" title="Principal component analysis">principal component analysis</a> which require atoms <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle d_{1},...,d_{n}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mi>d</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
</msub>
<mo>,</mo>
<mo>.</mo>
<mo>.</mo>
<mo>.</mo>
<mo>,</mo>
<msub>
<mi>d</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>n</mi>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle d_{1},...,d_{n}}</annotation>
</semantics>
</math></span><img src="./b3b89a8665e8b6ec19d0787a8f064a2d1a0024c3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:9.86ex; height:2.509ex;" alt="{\displaystyle d_{1},...,d_{n}}" loading="lazy"></span> to be orthogonal. The choice of these subspaces is crucial for efficient dimensionality reduction, but it is not trivial. And dimensionality reduction based on dictionary representation can be extended to address specific tasks such as data analysis or classification. However, their main downside is limiting the choice of atoms.
</p><p>Overcomplete dictionaries, however, do not require the atoms to be orthogonal (they will never have a <a href="Basis_(linear_algebra)" title="Basis (linear algebra)">basis</a> anyway) thus allowing for more flexible dictionaries and richer data representations.
</p><p>An overcomplete dictionary which allows for sparse representation of signal can be a famous transform matrix (wavelets transform, fourier transform) or it can be formulated so that its elements are changed in such a way that it sparsely represents the given signal in a best way. Learned dictionaries are capable of giving sparser solutions as compared to predefined transform matrices.
</p>
<div class="mw-heading mw-heading2"><h2 id="Algorithms">Algorithms</h2></div>
<p>As the optimization problem described above can be solved as a convex problem with respect to either dictionary or sparse coding while the other one of the two is fixed, most of the algorithms are based on the idea of iteratively updating one and then the other.
</p><p>The problem of finding an optimal sparse coding <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle R}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>R</mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle R}</annotation>
</semantics>
</math></span><img src="./4b0bfb3769bf24d80e15374dc37b0441e2616e33.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.764ex; height:2.176ex;" alt="{\displaystyle R}" loading="lazy"></span> with a given dictionary <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} }</annotation>
</semantics>
</math></span><img src="./b2345293072878db24e119c580def49ad582e3ed.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.05ex; height:2.176ex;" alt="{\displaystyle \mathbf {D} }" loading="lazy"></span> is known as <a href="Sparse_approximation" title="Sparse approximation">sparse approximation</a> (or sometimes just sparse coding problem). A number of algorithms have been developed to solve it (such as <a href="Matching_pursuit" title="Matching pursuit">matching pursuit</a> and <a href="Lasso_(statistics)" title="Lasso (statistics)">LASSO</a>) and are incorporated in the algorithms described below.
</p>
<div class="mw-heading mw-heading3"><h3 id="Method_of_optimal_directions_(MOD)">Method of optimal directions (MOD)</h3></div>
<p>The method of optimal directions (or MOD) was one of the first methods introduced to tackle the sparse dictionary learning problem.<sup id="cite_ref-5" class="reference"><a href="#cite_note-5"><span class="cite-bracket">[</span>5<span class="cite-bracket">]</span></a></sup> The core idea of it is to solve the minimization problem subject to the limited number of non-zero components of the representation vector:
</p><p><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \min _{\mathbf {D} ,R}\{\|X-\mathbf {D} R\|_{F}^{2}\}\,\,{\text{s.t.}}\,\,\forall i\,\,\|r_{i}\|_{0}\leq T}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<munder>
<mo movablelimits="true" form="prefix">min</mo>
<mrow class="MJX-TeXAtom-ORD">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mo>,</mo>
<mi>R</mi>
</mrow>
</munder>
<mo fence="false" stretchy="false">{</mo>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mi>X</mi>
<mo>−<!-- − --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mi>R</mi>
<msubsup>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>F</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo fence="false" stretchy="false">}</mo>
<mspace width="thinmathspace"></mspace>
<mspace width="thinmathspace"></mspace>
<mrow class="MJX-TeXAtom-ORD">
<mtext>s.t.</mtext>
</mrow>
<mspace width="thinmathspace"></mspace>
<mspace width="thinmathspace"></mspace>
<mi mathvariant="normal">∀<!-- ∀ --></mi>
<mi>i</mi>
<mspace width="thinmathspace"></mspace>
<mspace width="thinmathspace"></mspace>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<msub>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mn>0</mn>
</mrow>
</msub>
<mo>≤<!-- ≤ --></mo>
<mi>T</mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \min _{\mathbf {D} ,R}\{\|X-\mathbf {D} R\|_{F}^{2}\}\,\,{\text{s.t.}}\,\,\forall i\,\,\|r_{i}\|_{0}\leq T}</annotation>
</semantics>
</math></span><img src="./a5969b918bfcef6fec905764c827db5ca78b0f7c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.338ex; width:36.117ex; height:4.509ex;" alt="{\displaystyle \min _{\mathbf {D} ,R}\{\|X-\mathbf {D} R\|_{F}^{2}\}\,\,{\text{s.t.}}\,\,\forall i\,\,\|r_{i}\|_{0}\leq T}" loading="lazy"></span>
</p><p>Here, <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle F}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>F</mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle F}</annotation>
</semantics>
</math></span><img src="./545fd099af8541605f7ee55f08225526be88ce57.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.741ex; height:2.176ex;" alt="{\displaystyle F}" loading="lazy"></span> denotes the <a href="Frobenius_norm" class="mw-redirect" title="Frobenius norm">Frobenius norm</a>. MOD alternates between getting the <a href="Sparse_approximation" title="Sparse approximation">sparse coding</a> using a method such as <a href="Matching_pursuit" title="Matching pursuit">matching pursuit</a> and updating the dictionary by computing the analytical solution of the problem given by <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} =XR^{+}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mo>=</mo>
<mi>X</mi>
<msup>
<mi>R</mi>
<mrow class="MJX-TeXAtom-ORD">
<mo>+</mo>
</mrow>
</msup>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} =XR^{+}}</annotation>
</semantics>
</math></span><img src="./414b5a914ef85da13a21354d33a17d4d64ee96dd.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:10.403ex; height:2.509ex;" alt="{\displaystyle \mathbf {D} =XR^{+}}" loading="lazy"></span> where <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle R^{+}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msup>
<mi>R</mi>
<mrow class="MJX-TeXAtom-ORD">
<mo>+</mo>
</mrow>
</msup>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle R^{+}}</annotation>
</semantics>
</math></span><img src="./86bba4b120e2b4fcffb404952d8965923b481380.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:3.275ex; height:2.509ex;" alt="{\displaystyle R^{+}}" loading="lazy"></span> is a <a href="Moore%E2%80%93Penrose_pseudoinverse" class="mw-redirect" title="Moore–Penrose pseudoinverse">Moore-Penrose pseudoinverse</a>. After this update <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} }</annotation>
</semantics>
</math></span><img src="./b2345293072878db24e119c580def49ad582e3ed.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.05ex; height:2.176ex;" alt="{\displaystyle \mathbf {D} }" loading="lazy"></span> is renormalized to fit the constraints and the new sparse coding is obtained again. The process is repeated until convergence (or until a sufficiently small residue).
</p><p>MOD has proved to be a very efficient method for low-dimensional input data <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>X</mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle X}</annotation>
</semantics>
</math></span><img src="./68baa052181f707c662844a465bfeeb135e82bab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\displaystyle X}" loading="lazy"></span> requiring just a few iterations to converge. However, due to the high complexity of the matrix-inversion operation, computing the pseudoinverse in high-dimensional cases is in many cases intractable. This shortcoming has inspired the development of other dictionary learning methods.
</p>
<div class="mw-heading mw-heading3"><h3 id="K-SVD">K-SVD</h3></div>
<style data-mw-deduplicate="TemplateStyles:r1236090951">
/* start https://en.wikipedia.org/ */
.mw-parser-output .hatnote{font-style:italic}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output .hatnote i{font-style:normal}.mw-parser-output .hatnote+link+.hatnote{margin-top:-0.5em}@media print{body.ns-0 .mw-parser-output .hatnote{display:none!important}}
/* end https://en.wikipedia.org/ */
</style><div role="note" class="hatnote navigation-not-searchable">Main article: <a href="K-SVD" title="K-SVD">K-SVD</a></div><p><a href="K-SVD" title="K-SVD">K-SVD</a> is an algorithm that performs <a href="Singular_value_decomposition" title="Singular value decomposition">SVD</a> at its core to update the atoms of the dictionary one by one and basically is a generalization of <a href="K-means_clustering" title="K-means clustering">K-means</a>. It enforces that each element of the input data <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x_{i}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle x_{i}}</annotation>
</semantics>
</math></span><img src="./e87000dd6142b81d041896a30fe58f0c3acb2158.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.129ex; height:2.009ex;" alt="{\displaystyle x_{i}}" loading="lazy"></span> is encoded by a linear combination of not more than <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle T_{0}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mi>T</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>0</mn>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle T_{0}}</annotation>
</semantics>
</math></span><img src="./55b9e7d7b96196b5a6a26f4349caa3ac82fd67e3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.412ex; height:2.509ex;" alt="{\displaystyle T_{0}}" loading="lazy"></span> elements in a way identical to the MOD approach:
</p><p><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \min _{\mathbf {D} ,R}\{\|X-\mathbf {D} R\|_{F}^{2}\}\,\,{\text{s.t.}}\,\,\forall i\,\,\|r_{i}\|_{0}\leq T_{0}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<munder>
<mo movablelimits="true" form="prefix">min</mo>
<mrow class="MJX-TeXAtom-ORD">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mo>,</mo>
<mi>R</mi>
</mrow>
</munder>
<mo fence="false" stretchy="false">{</mo>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mi>X</mi>
<mo>−<!-- − --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mi>R</mi>
<msubsup>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>F</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo fence="false" stretchy="false">}</mo>
<mspace width="thinmathspace"></mspace>
<mspace width="thinmathspace"></mspace>
<mrow class="MJX-TeXAtom-ORD">
<mtext>s.t.</mtext>
</mrow>
<mspace width="thinmathspace"></mspace>
<mspace width="thinmathspace"></mspace>
<mi mathvariant="normal">∀<!-- ∀ --></mi>
<mi>i</mi>
<mspace width="thinmathspace"></mspace>
<mspace width="thinmathspace"></mspace>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<msub>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mn>0</mn>
</mrow>
</msub>
<mo>≤<!-- ≤ --></mo>
<msub>
<mi>T</mi>
<mrow class="MJX-TeXAtom-ORD">
<mn>0</mn>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \min _{\mathbf {D} ,R}\{\|X-\mathbf {D} R\|_{F}^{2}\}\,\,{\text{s.t.}}\,\,\forall i\,\,\|r_{i}\|_{0}\leq T_{0}}</annotation>
</semantics>
</math></span><img src="./f364303e02f4afdb29a734898b136cb20b094d81.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.338ex; width:36.892ex; height:4.509ex;" alt="{\displaystyle \min _{\mathbf {D} ,R}\{\|X-\mathbf {D} R\|_{F}^{2}\}\,\,{\text{s.t.}}\,\,\forall i\,\,\|r_{i}\|_{0}\leq T_{0}}" loading="lazy"></span>
</p><p>This algorithm's essence is to first fix the dictionary, find the best possible <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle R}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>R</mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle R}</annotation>
</semantics>
</math></span><img src="./4b0bfb3769bf24d80e15374dc37b0441e2616e33.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.764ex; height:2.176ex;" alt="{\displaystyle R}" loading="lazy"></span> under the above constraint (using <a href="Matching_pursuit#Extensions" title="Matching pursuit">Orthogonal Matching Pursuit</a>) and then iteratively update the atoms of dictionary <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} }</annotation>
</semantics>
</math></span><img src="./b2345293072878db24e119c580def49ad582e3ed.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.05ex; height:2.176ex;" alt="{\displaystyle \mathbf {D} }" loading="lazy"></span> in the following manner:
</p><p><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \|X-\mathbf {D} R\|_{F}^{2}=\left|X-\sum _{i=1}^{K}d_{i}x_{T}^{i}\right|_{F}^{2}=\|E_{k}-d_{k}x_{T}^{k}\|_{F}^{2}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mi>X</mi>
<mo>−<!-- − --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mi>R</mi>
<msubsup>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>F</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo>=</mo>
<msubsup>
<mrow>
<mo>|</mo>
<mrow>
<mi>X</mi>
<mo>−<!-- − --></mo>
<munderover>
<mo>∑<!-- ∑ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>K</mi>
</mrow>
</munderover>
<msub>
<mi>d</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<msubsup>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>T</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msubsup>
</mrow>
<mo>|</mo>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>F</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo>=</mo>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<msub>
<mi>E</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>k</mi>
</mrow>
</msub>
<mo>−<!-- − --></mo>
<msub>
<mi>d</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>k</mi>
</mrow>
</msub>
<msubsup>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>T</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>k</mi>
</mrow>
</msubsup>
<msubsup>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>F</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \|X-\mathbf {D} R\|_{F}^{2}=\left|X-\sum _{i=1}^{K}d_{i}x_{T}^{i}\right|_{F}^{2}=\|E_{k}-d_{k}x_{T}^{k}\|_{F}^{2}}</annotation>
</semantics>
</math></span><img src="./4894b688799f3f2f8db7937b7b354e37bb79fc1d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.171ex; width:49.115ex; height:7.843ex;" alt="{\displaystyle \|X-\mathbf {D} R\|_{F}^{2}=\left|X-\sum _{i=1}^{K}d_{i}x_{T}^{i}\right|_{F}^{2}=\|E_{k}-d_{k}x_{T}^{k}\|_{F}^{2}}" loading="lazy"></span>
</p><p>The next steps of the algorithm include <a href="Low-rank_approximation" title="Low-rank approximation">rank-1 approximation</a> of the residual matrix <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle E_{k}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mi>E</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>k</mi>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle E_{k}}</annotation>
</semantics>
</math></span><img src="./7587849b44d775263271e89499f4327eeac5dc81.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.804ex; height:2.509ex;" alt="{\displaystyle E_{k}}" loading="lazy"></span>, updating <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle d_{k}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mi>d</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>k</mi>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle d_{k}}</annotation>
</semantics>
</math></span><img src="./b78f5b2abc48e63b987b6d7527caa5aa9b1bb512.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.298ex; height:2.509ex;" alt="{\displaystyle d_{k}}" loading="lazy"></span> and enforcing the sparsity of <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x_{k}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>k</mi>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle x_{k}}</annotation>
</semantics>
</math></span><img src="./6d2b88c64c76a03611549fb9b4cf4ed060b56002.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.418ex; height:2.009ex;" alt="{\displaystyle x_{k}}" loading="lazy"></span> after the update. This algorithm is considered to be standard for dictionary learning and is used in a variety of applications. However, it shares weaknesses with MOD being efficient only for signals with relatively low dimensionality and having the possibility for being stuck at local minima.
</p>
<div class="mw-heading mw-heading3"><h3 id="Stochastic_gradient_descent">Stochastic gradient descent</h3></div>
<div role="note" class="hatnote navigation-not-searchable">Main article: <a href="Stochastic_gradient_descent" title="Stochastic gradient descent">Stochastic gradient descent</a></div><p>One can also apply a widespread stochastic gradient descent method with iterative projection to solve this problem.<sup id="cite_ref-6" class="reference"><a href="#cite_note-6"><span class="cite-bracket">[</span>6<span class="cite-bracket">]</span></a></sup> The idea of this method is to update the dictionary using the first order stochastic gradient and project it on the constraint set <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {C}}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mrow class="MJX-TeXAtom-ORD">
<mi class="MJX-tex-caligraphic" mathvariant="script">C</mi>
</mrow>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle {\mathcal {C}}}</annotation>
</semantics>
</math></span><img src="./e7b3edab7022ca9e2976651bc59c489513ee9019.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.239ex; height:2.176ex;" alt="{\displaystyle {\mathcal {C}}}" loading="lazy"></span>. The step that occurs at i-th iteration is described by this expression:
</p><p><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} _{i}={\text{proj}}_{\mathcal {C}}\left\{\mathbf {D} _{i-1}-\delta _{i}\nabla _{\mathbf {D} }\sum _{i\in S}\|x_{i}-\mathbf {D} r_{i}\|_{2}^{2}+\lambda \|r_{i}\|_{1}\right\}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<mo>=</mo>
<msub>
<mrow class="MJX-TeXAtom-ORD">
<mtext>proj</mtext>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mrow class="MJX-TeXAtom-ORD">
<mi class="MJX-tex-caligraphic" mathvariant="script">C</mi>
</mrow>
</mrow>
</msub>
<mrow>
<mo>{</mo>
<mrow>
<msub>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
<mo>−<!-- − --></mo>
<mn>1</mn>
</mrow>
</msub>
<mo>−<!-- − --></mo>
<msub>
<mi>δ<!-- δ --></mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<msub>
<mi mathvariant="normal">∇<!-- ∇ --></mi>
<mrow class="MJX-TeXAtom-ORD">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
</mrow>
</msub>
<munder>
<mo>∑<!-- ∑ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
<mo>∈<!-- ∈ --></mo>
<mi>S</mi>
</mrow>
</munder>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<mo>−<!-- − --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<msubsup>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo>+</mo>
<mi>λ<!-- λ --></mi>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<msub>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
</msub>
</mrow>
<mo>}</mo>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} _{i}={\text{proj}}_{\mathcal {C}}\left\{\mathbf {D} _{i-1}-\delta _{i}\nabla _{\mathbf {D} }\sum _{i\in S}\|x_{i}-\mathbf {D} r_{i}\|_{2}^{2}+\lambda \|r_{i}\|_{1}\right\}}</annotation>
</semantics>
</math></span><img src="./e91f6977f7bb5a6d98971a73984c843d070f4bbb.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.171ex; width:54.307ex; height:7.509ex;" alt="{\displaystyle \mathbf {D} _{i}={\text{proj}}_{\mathcal {C}}\left\{\mathbf {D} _{i-1}-\delta _{i}\nabla _{\mathbf {D} }\sum _{i\in S}\|x_{i}-\mathbf {D} r_{i}\|_{2}^{2}+\lambda \|r_{i}\|_{1}\right\}}" loading="lazy"></span>, where <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle S}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>S</mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle S}</annotation>
</semantics>
</math></span><img src="./4611d85173cd3b508e67077d4a1252c9c05abca2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.499ex; height:2.176ex;" alt="{\displaystyle S}" loading="lazy"></span> is a random subset of <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \{1...K\}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mo fence="false" stretchy="false">{</mo>
<mn>1...</mn>
<mi>K</mi>
<mo fence="false" stretchy="false">}</mo>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \{1...K\}}</annotation>
</semantics>
</math></span><img src="./a1814ace4a60b175597d7fc9c6f8321015a7ce4f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:7.494ex; height:2.843ex;" alt="{\displaystyle \{1...K\}}" loading="lazy"></span> and <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \delta _{i}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mi>δ<!-- δ --></mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \delta _{i}}</annotation>
</semantics>
</math></span><img src="./e0c5e905acee9cc0cf8bc01c08a4e876f43d5c1b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.832ex; height:2.676ex;" alt="{\displaystyle \delta _{i}}" loading="lazy"></span> is a gradient step.
</p>
<div class="mw-heading mw-heading3"><h3 id="Lagrange_dual_method">Lagrange dual method</h3></div>
<p>An algorithm based on solving a <a href="Duality_(optimization)" title="Duality (optimization)">dual Lagrangian problem</a> provides an efficient way to solve for the dictionary having no complications induced by the sparsity function.<sup id="cite_ref-7" class="reference"><a href="#cite_note-7"><span class="cite-bracket">[</span>7<span class="cite-bracket">]</span></a></sup> Consider the following Lagrangian:
</p><p><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {L}}(\mathbf {D} ,\Lambda )={\text{tr}}\left((X-\mathbf {D} R)^{T}(X-\mathbf {D} R)\right)+\sum _{j=1}^{n}\lambda _{j}\left({\sum _{i=1}^{d}\mathbf {D} _{ij}^{2}-c}\right)}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mrow class="MJX-TeXAtom-ORD">
<mi class="MJX-tex-caligraphic" mathvariant="script">L</mi>
</mrow>
</mrow>
<mo stretchy="false">(</mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mo>,</mo>
<mi mathvariant="normal">Λ<!-- Λ --></mi>
<mo stretchy="false">)</mo>
<mo>=</mo>
<mrow class="MJX-TeXAtom-ORD">
<mtext>tr</mtext>
</mrow>
<mrow>
<mo>(</mo>
<mrow>
<mo stretchy="false">(</mo>
<mi>X</mi>
<mo>−<!-- − --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mi>R</mi>
<msup>
<mo stretchy="false">)</mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>T</mi>
</mrow>
</msup>
<mo stretchy="false">(</mo>
<mi>X</mi>
<mo>−<!-- − --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mi>R</mi>
<mo stretchy="false">)</mo>
</mrow>
<mo>)</mo>
</mrow>
<mo>+</mo>
<munderover>
<mo>∑<!-- ∑ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>j</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>n</mi>
</mrow>
</munderover>
<msub>
<mi>λ<!-- λ --></mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>j</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mrow class="MJX-TeXAtom-ORD">
<munderover>
<mo>∑<!-- ∑ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>d</mi>
</mrow>
</munderover>
<msubsup>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
<mi>j</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo>−<!-- − --></mo>
<mi>c</mi>
</mrow>
<mo>)</mo>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle {\mathcal {L}}(\mathbf {D} ,\Lambda )={\text{tr}}\left((X-\mathbf {D} R)^{T}(X-\mathbf {D} R)\right)+\sum _{j=1}^{n}\lambda _{j}\left({\sum _{i=1}^{d}\mathbf {D} _{ij}^{2}-c}\right)}</annotation>
</semantics>
</math></span><img src="./96860b2e89b3b2b541cfb026b345e3e6b68a243f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.338ex; width:61.849ex; height:7.676ex;" alt="{\displaystyle {\mathcal {L}}(\mathbf {D} ,\Lambda )={\text{tr}}\left((X-\mathbf {D} R)^{T}(X-\mathbf {D} R)\right)+\sum _{j=1}^{n}\lambda _{j}\left({\sum _{i=1}^{d}\mathbf {D} _{ij}^{2}-c}\right)}" loading="lazy"></span>, where <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle c}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>c</mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle c}</annotation>
</semantics>
</math></span><img src="./86a67b81c2de995bd608d5b2df50cd8cd7d92455.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.007ex; height:1.676ex;" alt="{\displaystyle c}" loading="lazy"></span> is a constraint on the norm of the atoms and <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \lambda _{i}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mi>λ<!-- λ --></mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \lambda _{i}}</annotation>
</semantics>
</math></span><img src="./72fde940918edf84caf3d406cc7d31949166820f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.155ex; height:2.509ex;" alt="{\displaystyle \lambda _{i}}" loading="lazy"></span> are the so-called dual variables forming the diagonal matrix <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Lambda }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi mathvariant="normal">Λ<!-- Λ --></mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \Lambda }</annotation>
</semantics>
</math></span><img src="./0ac0a4a98a414e3480335f9ba652d12571ec6733.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.613ex; height:2.176ex;" alt="{\displaystyle \Lambda }" loading="lazy"></span>.
</p><p>We can then provide an analytical expression for the Lagrange dual after minimization over <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} }</annotation>
</semantics>
</math></span><img src="./b2345293072878db24e119c580def49ad582e3ed.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.05ex; height:2.176ex;" alt="{\displaystyle \mathbf {D} }" loading="lazy"></span>:
</p><p><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {D}}(\Lambda )=\min _{\mathbf {D} }{\mathcal {L}}(\mathbf {D} ,\Lambda )={\text{tr}}(X^{T}X-XR^{T}(RR^{T}+\Lambda )^{-1}(XR^{T})^{T}-c\Lambda )}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mrow class="MJX-TeXAtom-ORD">
<mi class="MJX-tex-caligraphic" mathvariant="script">D</mi>
</mrow>
</mrow>
<mo stretchy="false">(</mo>
<mi mathvariant="normal">Λ<!-- Λ --></mi>
<mo stretchy="false">)</mo>
<mo>=</mo>
<munder>
<mo movablelimits="true" form="prefix">min</mo>
<mrow class="MJX-TeXAtom-ORD">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
</mrow>
</munder>
<mrow class="MJX-TeXAtom-ORD">
<mrow class="MJX-TeXAtom-ORD">
<mi class="MJX-tex-caligraphic" mathvariant="script">L</mi>
</mrow>
</mrow>
<mo stretchy="false">(</mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mo>,</mo>
<mi mathvariant="normal">Λ<!-- Λ --></mi>
<mo stretchy="false">)</mo>
<mo>=</mo>
<mrow class="MJX-TeXAtom-ORD">
<mtext>tr</mtext>
</mrow>
<mo stretchy="false">(</mo>
<msup>
<mi>X</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>T</mi>
</mrow>
</msup>
<mi>X</mi>
<mo>−<!-- − --></mo>
<mi>X</mi>
<msup>
<mi>R</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>T</mi>
</mrow>
</msup>
<mo stretchy="false">(</mo>
<mi>R</mi>
<msup>
<mi>R</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>T</mi>
</mrow>
</msup>
<mo>+</mo>
<mi mathvariant="normal">Λ<!-- Λ --></mi>
<msup>
<mo stretchy="false">)</mo>
<mrow class="MJX-TeXAtom-ORD">
<mo>−<!-- − --></mo>
<mn>1</mn>
</mrow>
</msup>
<mo stretchy="false">(</mo>
<mi>X</mi>
<msup>
<mi>R</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>T</mi>
</mrow>
</msup>
<msup>
<mo stretchy="false">)</mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>T</mi>
</mrow>
</msup>
<mo>−<!-- − --></mo>
<mi>c</mi>
<mi mathvariant="normal">Λ<!-- Λ --></mi>
<mo stretchy="false">)</mo>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle {\mathcal {D}}(\Lambda )=\min _{\mathbf {D} }{\mathcal {L}}(\mathbf {D} ,\Lambda )={\text{tr}}(X^{T}X-XR^{T}(RR^{T}+\Lambda )^{-1}(XR^{T})^{T}-c\Lambda )}</annotation>
</semantics>
</math></span><img src="./7f2f50090df545befbca3ffbc303659130be83ae.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.005ex; width:68.053ex; height:4.343ex;" alt="{\displaystyle {\mathcal {D}}(\Lambda )=\min _{\mathbf {D} }{\mathcal {L}}(\mathbf {D} ,\Lambda )={\text{tr}}(X^{T}X-XR^{T}(RR^{T}+\Lambda )^{-1}(XR^{T})^{T}-c\Lambda )}" loading="lazy"></span>.
</p><p>After applying one of the optimization methods to the value of the dual (such as <a href="Newton's_method_in_optimization" title="Newton's method in optimization">Newton's method</a> or <a href="Conjugate_gradient_method" title="Conjugate gradient method">conjugate gradient</a>) we get the value of <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} }</annotation>
</semantics>
</math></span><img src="./b2345293072878db24e119c580def49ad582e3ed.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.05ex; height:2.176ex;" alt="{\displaystyle \mathbf {D} }" loading="lazy"></span>:
</p><p><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} ^{T}=(RR^{T}+\Lambda )^{-1}(XR^{T})^{T}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msup>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>T</mi>
</mrow>
</msup>
<mo>=</mo>
<mo stretchy="false">(</mo>
<mi>R</mi>
<msup>
<mi>R</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>T</mi>
</mrow>
</msup>
<mo>+</mo>
<mi mathvariant="normal">Λ<!-- Λ --></mi>
<msup>
<mo stretchy="false">)</mo>
<mrow class="MJX-TeXAtom-ORD">
<mo>−<!-- − --></mo>
<mn>1</mn>
</mrow>
</msup>
<mo stretchy="false">(</mo>
<mi>X</mi>
<msup>
<mi>R</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>T</mi>
</mrow>
</msup>
<msup>
<mo stretchy="false">)</mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>T</mi>
</mrow>
</msup>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} ^{T}=(RR^{T}+\Lambda )^{-1}(XR^{T})^{T}}</annotation>
</semantics>
</math></span><img src="./d1dc6bb18909e71a3cadb0617e83ce6a04cde07e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:28.382ex; height:3.176ex;" alt="{\displaystyle \mathbf {D} ^{T}=(RR^{T}+\Lambda )^{-1}(XR^{T})^{T}}" loading="lazy"></span>
</p><p>Solving this problem is less computational hard because the amount of dual variables <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>n</mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle n}</annotation>
</semantics>
</math></span><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy"></span> is a lot of times much less than the amount of variables in the primal problem.
</p>
<div class="mw-heading mw-heading3"><h3 id="LASSO">LASSO</h3></div>
<div role="note" class="hatnote navigation-not-searchable">Main article: <a href="Lasso_(statistics)" title="Lasso (statistics)">Lasso (statistics)</a></div>
<p>In this approach, the optimization problem is formulated as:
</p><p><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \min _{r\in \mathbb {R} ^{n}}\{\,\,\|r\|_{1}\}\,\,{\text{subject to}}\,\,\|X-\mathbf {D} R\|_{F}^{2}<\epsilon }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<munder>
<mo movablelimits="true" form="prefix">min</mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>r</mi>
<mo>∈<!-- ∈ --></mo>
<msup>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="double-struck">R</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>n</mi>
</mrow>
</msup>
</mrow>
</munder>
<mo fence="false" stretchy="false">{</mo>
<mspace width="thinmathspace"></mspace>
<mspace width="thinmathspace"></mspace>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mi>r</mi>
<msub>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
</msub>
<mo fence="false" stretchy="false">}</mo>
<mspace width="thinmathspace"></mspace>
<mspace width="thinmathspace"></mspace>
<mrow class="MJX-TeXAtom-ORD">
<mtext>subject to</mtext>
</mrow>
<mspace width="thinmathspace"></mspace>
<mspace width="thinmathspace"></mspace>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mi>X</mi>
<mo>−<!-- − --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mi>R</mi>
<msubsup>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>F</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo><</mo>
<mi>ϵ<!-- ϵ --></mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \min _{r\in \mathbb {R} ^{n}}\{\,\,\|r\|_{1}\}\,\,{\text{subject to}}\,\,\|X-\mathbf {D} R\|_{F}^{2}<\epsilon }</annotation>
</semantics>
</math></span><img src="./b2723d6a02e7e8cdf2bdd1d6495c2589d83ff88e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.171ex; width:39.36ex; height:4.343ex;" alt="{\displaystyle \min _{r\in \mathbb {R} ^{n}}\{\,\,\|r\|_{1}\}\,\,{\text{subject to}}\,\,\|X-\mathbf {D} R\|_{F}^{2}<\epsilon }" loading="lazy"></span>, where <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \epsilon }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>ϵ<!-- ϵ --></mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \epsilon }</annotation>
</semantics>
</math></span><img src="./c3837cad72483d97bcdde49c85d3b7b859fb3fd2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:0.944ex; height:1.676ex;" alt="{\displaystyle \epsilon }" loading="lazy"></span> is the permitted error in the reconstruction LASSO.
</p><p>It finds an estimate of <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle r_{i}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle r_{i}}</annotation>
</semantics>
</math></span><img src="./a0b6d651eaf432dbf1f106021c8bb499ae83fd1f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.848ex; height:2.009ex;" alt="{\displaystyle r_{i}}" loading="lazy"></span> by minimizing the least square error subject to a <i><a href="L1-norm" class="mw-redirect" title="L1-norm">L</a></i><sup><a href="L1-norm" class="mw-redirect" title="L1-norm">1</a></sup><a href="L1-norm" class="mw-redirect" title="L1-norm">-norm</a> constraint in the solution vector, formulated as:
</p><p><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \min _{r\in \mathbb {R} ^{n}}\,\,{\dfrac {1}{2}}\,\,\|X-\mathbf {D} r\|_{F}^{2}+\lambda \,\,\|r\|_{1}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<munder>
<mo movablelimits="true" form="prefix">min</mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>r</mi>
<mo>∈<!-- ∈ --></mo>
<msup>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="double-struck">R</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>n</mi>
</mrow>
</msup>
</mrow>
</munder>
<mspace width="thinmathspace"></mspace>
<mspace width="thinmathspace"></mspace>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mfrac>
<mn>1</mn>
<mn>2</mn>
</mfrac>
</mstyle>
</mrow>
<mspace width="thinmathspace"></mspace>
<mspace width="thinmathspace"></mspace>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mi>X</mi>
<mo>−<!-- − --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mi>r</mi>
<msubsup>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>F</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo>+</mo>
<mi>λ<!-- λ --></mi>
<mspace width="thinmathspace"></mspace>
<mspace width="thinmathspace"></mspace>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mi>r</mi>
<msub>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \min _{r\in \mathbb {R} ^{n}}\,\,{\dfrac {1}{2}}\,\,\|X-\mathbf {D} r\|_{F}^{2}+\lambda \,\,\|r\|_{1}}</annotation>
</semantics>
</math></span><img src="./4b7b334ff1dd4f8675e099e0c57ab33cff045d9a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.171ex; width:29.028ex; height:5.509ex;" alt="{\displaystyle \min _{r\in \mathbb {R} ^{n}}\,\,{\dfrac {1}{2}}\,\,\|X-\mathbf {D} r\|_{F}^{2}+\lambda \,\,\|r\|_{1}}" loading="lazy"></span>, where <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \lambda >0}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>λ<!-- λ --></mi>
<mo>></mo>
<mn>0</mn>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \lambda >0}</annotation>
</semantics>
</math></span><img src="./eea25afc0351140f919cf791c49c1964b8b081de.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:5.616ex; height:2.176ex;" alt="{\displaystyle \lambda >0}" loading="lazy"></span> controls the trade-off between sparsity and the reconstruction error. This gives the global optimal solution.<sup id="cite_ref-8" class="reference"><a href="#cite_note-8"><span class="cite-bracket">[</span>8<span class="cite-bracket">]</span></a></sup> See also <a rel="nofollow" class="external text" href="https://www.di.ens.fr/~fbach/mairal_icml09.pdf">Online dictionary learning for Sparse coding</a>
</p>
<div class="mw-heading mw-heading3"><h3 id="Parametric_training_methods">Parametric training methods</h3></div>
<p>Parametric training methods are aimed to incorporate the best of both worlds — the realm of analytically constructed dictionaries and the learned ones.<sup id="cite_ref-9" class="reference"><a href="#cite_note-9"><span class="cite-bracket">[</span>9<span class="cite-bracket">]</span></a></sup> This allows to construct more powerful generalized dictionaries that can potentially be applied to the cases of arbitrary-sized signals. Notable approaches include:
</p>
<ul><li>Translation-invariant dictionaries.<sup id="cite_ref-10" class="reference"><a href="#cite_note-10"><span class="cite-bracket">[</span>10<span class="cite-bracket">]</span></a></sup> These dictionaries are composed by the translations of the atoms originating from the dictionary constructed for a finite-size signal patch. This allows the resulting dictionary to provide a representation for the arbitrary-sized signal.</li>
<li>Multiscale dictionaries.<sup id="cite_ref-11" class="reference"><a href="#cite_note-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup> This method focuses on constructing a dictionary that is composed of differently scaled dictionaries to improve sparsity.</li>
<li>Sparse dictionaries.<sup id="cite_ref-12" class="reference"><a href="#cite_note-12"><span class="cite-bracket">[</span>12<span class="cite-bracket">]</span></a></sup> This method focuses on not only providing a sparse representation but also constructing a sparse dictionary which is enforced by the expression <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} =\mathbf {B} \mathbf {A} }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mo>=</mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">B</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">A</mi>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} =\mathbf {B} \mathbf {A} }</annotation>
</semantics>
</math></span><img src="./f90c98969d8e38d85cebd5ad01ac5f150d71ae46.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:9.069ex; height:2.176ex;" alt="{\displaystyle \mathbf {D} =\mathbf {B} \mathbf {A} }" loading="lazy"></span> where <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {B} }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">B</mi>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {B} }</annotation>
</semantics>
</math></span><img src="./cafb0ef39b0f5ffa23c170aa7f7b4e718327c4d1.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.901ex; height:2.176ex;" alt="{\displaystyle \mathbf {B} }" loading="lazy"></span> is some pre-defined analytical dictionary with desirable properties such as fast computation and <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {A} }">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">A</mi>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {A} }</annotation>
</semantics>
</math></span><img src="./0795cc96c75d81520a120482662b90f024c9a1a1.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.019ex; height:2.176ex;" alt="{\displaystyle \mathbf {A} }" loading="lazy"></span> is a sparse matrix. Such formulation allows to directly combine the fast implementation of analytical dictionaries with the flexibility of sparse approaches.</li></ul>
<div class="mw-heading mw-heading3"><h3 id="Online_dictionary_learning_(LASSO_approach)">Online dictionary learning (<a rel="nofollow" class="external text" href="https://www.di.ens.fr/~fbach/mairal_icml09.pdf">LASSO approach</a>)</h3></div>
<p>Many common approaches to sparse dictionary learning rely on the fact that the whole input data <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>X</mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle X}</annotation>
</semantics>
</math></span><img src="./68baa052181f707c662844a465bfeeb135e82bab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\displaystyle X}" loading="lazy"></span> (or at least a large enough training dataset) is available for the algorithm. However, this might not be the case in the real-world scenario as the size of the input data might be too big to fit it into memory. The other case where this assumption can not be made is when the input data comes in a form of a <a href="Stream_(computing)" title="Stream (computing)">stream</a>. Such cases lie in the field of study of <a href="Online_machine_learning" title="Online machine learning">online learning</a> which essentially suggests iteratively updating the model upon the new data points <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>x</mi>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle x}</annotation>
</semantics>
</math></span><img src="./87f9e315fd7e2ba406057a97300593c4802b53e4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.33ex; height:1.676ex;" alt="{\displaystyle x}" loading="lazy"></span> becoming available.
</p><p>A dictionary can be learned in an online manner the following way:<sup id="cite_ref-13" class="reference"><a href="#cite_note-13"><span class="cite-bracket">[</span>13<span class="cite-bracket">]</span></a></sup>
</p>
<ol><li>For <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle t=1...T:}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<mi>t</mi>
<mo>=</mo>
<mn>1...</mn>
<mi>T</mi>
<mo>:</mo>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle t=1...T:}</annotation>
</semantics>
</math></span><img src="./c5c03b197932ce233789aa0e1db873882387ad55.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:9.969ex; height:2.176ex;" alt="{\displaystyle t=1...T:}" loading="lazy"></span></li>
<li>Draw a new sample <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x_{t}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>t</mi>
</mrow>
</msub>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle x_{t}}</annotation>
</semantics>
</math></span><img src="./f279a30bc8eabc788f3fe81c9cfb674e72e858db.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.156ex; height:2.009ex;" alt="{\displaystyle x_{t}}" loading="lazy"></span></li>
<li>Find a sparse coding using <a href="Least-angle_regression" title="Least-angle regression">LARS</a>: <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle r_{t}={\underset {r\in \mathbb {R} ^{n}}{\text{argmin}}}\left({\frac {1}{2}}\|x_{t}-\mathbf {D} _{t-1}r\|+\lambda \|r\|_{1}\right)}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>t</mi>
</mrow>
</msub>
<mo>=</mo>
<mrow class="MJX-TeXAtom-ORD">
<munder>
<mtext>argmin</mtext>
<mrow>
<mi>r</mi>
<mo>∈<!-- ∈ --></mo>
<msup>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="double-struck">R</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>n</mi>
</mrow>
</msup>
</mrow>
</munder>
</mrow>
<mrow>
<mo>(</mo>
<mrow>
<mrow class="MJX-TeXAtom-ORD">
<mfrac>
<mn>1</mn>
<mn>2</mn>
</mfrac>
</mrow>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>t</mi>
</mrow>
</msub>
<mo>−<!-- − --></mo>
<msub>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>t</mi>
<mo>−<!-- − --></mo>
<mn>1</mn>
</mrow>
</msub>
<mi>r</mi>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mo>+</mo>
<mi>λ<!-- λ --></mi>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mi>r</mi>
<msub>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
</msub>
</mrow>
<mo>)</mo>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle r_{t}={\underset {r\in \mathbb {R} ^{n}}{\text{argmin}}}\left({\frac {1}{2}}\|x_{t}-\mathbf {D} _{t-1}r\|+\lambda \|r\|_{1}\right)}</annotation>
</semantics>
</math></span><img src="./9459b74b076816441fa0bf50ca63b635e5884a0b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.671ex; width:39.861ex; height:6.343ex;" alt="{\displaystyle r_{t}={\underset {r\in \mathbb {R} ^{n}}{\text{argmin}}}\left({\frac {1}{2}}\|x_{t}-\mathbf {D} _{t-1}r\|+\lambda \|r\|_{1}\right)}" loading="lazy"></span></li>
<li>Update dictionary using <a href="Coordinate_descent" title="Coordinate descent">block-coordinate</a> approach: <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {D} _{t}={\underset {\mathbf {D} \in {\mathcal {C}}}{\text{argmin}}}{\frac {1}{t}}\sum _{i=1}^{t}\left({\frac {1}{2}}\|x_{i}-\mathbf {D} r_{i}\|_{2}^{2}+\lambda \|r_{i}\|_{1}\right)}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle displaystyle="true" scriptlevel="0">
<msub>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>t</mi>
</mrow>
</msub>
<mo>=</mo>
<mrow class="MJX-TeXAtom-ORD">
<munder>
<mtext>argmin</mtext>
<mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<mo>∈<!-- ∈ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mrow class="MJX-TeXAtom-ORD">
<mi class="MJX-tex-caligraphic" mathvariant="script">C</mi>
</mrow>
</mrow>
</mrow>
</munder>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mfrac>
<mn>1</mn>
<mi>t</mi>
</mfrac>
</mrow>
<munderover>
<mo>∑<!-- ∑ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>t</mi>
</mrow>
</munderover>
<mrow>
<mo>(</mo>
<mrow>
<mrow class="MJX-TeXAtom-ORD">
<mfrac>
<mn>1</mn>
<mn>2</mn>
</mfrac>
</mrow>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<msub>
<mi>x</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<mo>−<!-- − --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mi mathvariant="bold">D</mi>
</mrow>
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<msubsup>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mn>2</mn>
</mrow>
</msubsup>
<mo>+</mo>
<mi>λ<!-- λ --></mi>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<msub>
<mi>r</mi>
<mrow class="MJX-TeXAtom-ORD">
<mi>i</mi>
</mrow>
</msub>
<msub>
<mo fence="false" stretchy="false">‖<!-- ‖ --></mo>
<mrow class="MJX-TeXAtom-ORD">
<mn>1</mn>
</mrow>
</msub>
</mrow>
<mo>)</mo>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {D} _{t}={\underset {\mathbf {D} \in {\mathcal {C}}}{\text{argmin}}}{\frac {1}{t}}\sum _{i=1}^{t}\left({\frac {1}{2}}\|x_{i}-\mathbf {D} r_{i}\|_{2}^{2}+\lambda \|r_{i}\|_{1}\right)}</annotation>
</semantics>
</math></span><img src="./8fc47051c8b102e27c9199733e35ea5b38e50b60.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.005ex; width:46.303ex; height:7.176ex;" alt="{\displaystyle \mathbf {D} _{t}={\underset {\mathbf {D} \in {\mathcal {C}}}{\text{argmin}}}{\frac {1}{t}}\sum _{i=1}^{t}\left({\frac {1}{2}}\|x_{i}-\mathbf {D} r_{i}\|_{2}^{2}+\lambda \|r_{i}\|_{1}\right)}" loading="lazy"></span></li></ol>
<p>This method allows us to gradually update the dictionary as new data becomes available for sparse representation learning and helps drastically reduce the amount of memory needed to store the dataset (which often has a huge size).
</p>
<div class="mw-heading mw-heading2"><h2 id="Applications">Applications</h2></div>
<p>The dictionary learning framework, namely the linear decomposition of an input signal using a few basis elements learned from data itself, has led to state-of-art results in various image and video processing tasks. This technique can be applied to classification problems in a way that if we have built specific dictionaries for each class, the input signal can be classified by finding the dictionary corresponding to the sparsest representation.
It also has properties that are useful for signal denoising since usually one can learn a dictionary to represent the meaningful part of the input signal in a sparse way but the noise in the input will have a much less sparse representation.<sup id="cite_ref-14" class="reference"><a href="#cite_note-14"><span class="cite-bracket">[</span>14<span class="cite-bracket">]</span></a></sup>
</p><p>Sparse dictionary learning has been successfully applied to various image, video and audio processing tasks as well as to texture synthesis<sup id="cite_ref-15" class="reference"><a href="#cite_note-15"><span class="cite-bracket">[</span>15<span class="cite-bracket">]</span></a></sup> and unsupervised clustering.<sup id="cite_ref-16" class="reference"><a href="#cite_note-16"><span class="cite-bracket">[</span>16<span class="cite-bracket">]</span></a></sup> In evaluations with the <a href="Bag-of-words_model_in_computer_vision" title="Bag-of-words model in computer vision">Bag-of-Words</a> model,<sup id="cite_ref-17" class="reference"><a href="#cite_note-17"><span class="cite-bracket">[</span>17<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-18" class="reference"><a href="#cite_note-18"><span class="cite-bracket">[</span>18<span class="cite-bracket">]</span></a></sup> sparse coding was found empirically to outperform other coding approaches on the object category recognition tasks.
</p><p>Dictionary learning is used to analyse medical signals in detail. Such medical signals include those from electroencephalography (EEG), electrocardiography (ECG), magnetic resonance imaging (MRI), functional MRI (fMRI), continuous glucose monitors <sup id="cite_ref-19" class="reference"><a href="#cite_note-19"><span class="cite-bracket">[</span>19<span class="cite-bracket">]</span></a></sup> and ultrasound computer tomography (USCT), where different assumptions are used to analyze each signal.
</p>
<div class="mw-heading mw-heading2"><h2 id="See_also">See also</h2></div>
<ul><li><a href="Sparse_approximation" title="Sparse approximation">Sparse approximation</a></li>
<li><a href="Sparse_PCA" title="Sparse PCA">Sparse PCA</a></li>
<li><a href="K-SVD" title="K-SVD">K-SVD</a></li>
<li><a href="Matrix_factorization" class="mw-redirect" title="Matrix factorization">Matrix factorization</a></li>
<li><a href="Sparse_coding" class="mw-redirect" title="Sparse coding">Neural sparse coding</a></li></ul>
<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>
<div class="mw-references-wrap mw-references-columns"><ol class="references">
<li id="cite_note-1"><span class="mw-cite-backlink"><b><a href="#cite_ref-1">^</a></b></span> <span class="reference-text"><style data-mw-deduplicate="TemplateStyles:r1238218222">
/* start https://en.wikipedia.org/ */
.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}
/* end https://en.wikipedia.org/ */
</style><cite id="CITEREFNeedellTropp2009" class="citation journal cs1">Needell, D.; Tropp, J.A. (2009). "CoSaMP: Iterative signal recovery from incomplete and inaccurate samples". <i>Applied and Computational Harmonic Analysis</i>. <b>26</b> (3): <span class="nowrap">301–</span>321. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/0803.2392">0803.2392</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.acha.2008.07.002">10.1016/j.acha.2008.07.002</a>.</cite></span>
</li>
<li id="cite_note-2"><span class="mw-cite-backlink"><b><a href="#cite_ref-2">^</a></b></span> <span class="reference-text">Lotfi, M.; Vidyasagar, M."<a href="https://arxiv.org/abs/1708.03608" class="extiw external" title="arxiv:1708.03608">A Fast Non-iterative Algorithm for Compressive Sensing Using Binary Measurement Matrices</a>"</span>
</li>
<li id="cite_note-3"><span class="mw-cite-backlink"><b><a href="#cite_ref-3">^</a></b></span> <span class="reference-text">A. M. Tillmann, "<a href="https://doi.org/10.1109/LSP.2014.2345761" class="extiw external" title="doi:10.1109/LSP.2014.2345761">On the Computational Intractability of Exact and Approximate Dictionary Learning</a>", IEEE Signal Processing Letters 22(1), 2015: 45–49.</span>
</li>
<li id="cite_note-4"><span class="mw-cite-backlink"><b><a href="#cite_ref-4">^</a></b></span> <span class="reference-text"><cite id="CITEREFDonoho2006" class="citation journal cs1">Donoho, David L. (2006-06-01). "For most large underdetermined systems of linear equations the minimal 𝓁1-norm solution is also the sparsest solution". <i>Communications on Pure and Applied Mathematics</i>. <b>59</b> (6): <span class="nowrap">797–</span>829. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1002%2Fcpa.20132">10.1002/cpa.20132</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1097-0312">1097-0312</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:8510060">8510060</a>.</cite></span>
</li>
<li id="cite_note-5"><span class="mw-cite-backlink"><b><a href="#cite_ref-5">^</a></b></span> <span class="reference-text"><cite id="CITEREFEnganAaseHakon_Husoy1999" class="citation book cs1"><a href="Kjersti_Engan" title="Kjersti Engan">Engan, K.</a>; Aase, S.O.; Hakon Husoy, J. (1999-01-01). "Method of optimal directions for frame design". <i>1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258)</i>. Vol. 5. pp. 2443–2446 vol.5. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FICASSP.1999.760624">10.1109/ICASSP.1999.760624</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-0-7803-5041-0</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:33097614">33097614</a>.</cite></span>
</li>
<li id="cite_note-6"><span class="mw-cite-backlink"><b><a href="#cite_ref-6">^</a></b></span> <span class="reference-text"><cite id="CITEREFAharonElad2008" class="citation journal cs1"><a href="Michal_Aharon" title="Michal Aharon">Aharon, Michal</a>; Elad, Michael (2008). "Sparse and Redundant Modeling of Image Content Using an Image-Signature-Dictionary". <i>SIAM Journal on Imaging Sciences</i>. <b>1</b> (3): <span class="nowrap">228–</span>247. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.298.6982">10.1.1.298.6982</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1137%2F07070156x">10.1137/07070156x</a>.</cite></span>
</li>
<li id="cite_note-7"><span class="mw-cite-backlink"><b><a href="#cite_ref-7">^</a></b></span> <span class="reference-text">Lee, Honglak, et al. "Efficient sparse coding algorithms." <i>Advances in neural information processing systems</i>. 2006.</span>
</li>
<li id="cite_note-8"><span class="mw-cite-backlink"><b><a href="#cite_ref-8">^</a></b></span> <span class="reference-text"><cite id="CITEREFKumarKataria" class="citation web cs1">Kumar, Abhay; Kataria, Saurabh. <a rel="nofollow" class="external text" href="http://home.iitk.ac.in/~saurabhk/EE609A_12011_12807637_.pdf">"Dictionary Learning Based Applications in Image Processing using Convex Optimisation"</a> <span class="cs1-format">(PDF)</span>.</cite></span>
</li>
<li id="cite_note-9"><span class="mw-cite-backlink"><b><a href="#cite_ref-9">^</a></b></span> <span class="reference-text"><cite id="CITEREFRubinsteinBrucksteinElad2010" class="citation journal cs1">Rubinstein, R.; Bruckstein, A.M.; Elad, M. (2010-06-01). "Dictionaries for Sparse Representation Modeling". <i>Proceedings of the IEEE</i>. <b>98</b> (6): <span class="nowrap">1045–</span>1057. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.160.527">10.1.1.160.527</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FJPROC.2010.2040551">10.1109/JPROC.2010.2040551</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/0018-9219">0018-9219</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:2176046">2176046</a>.</cite></span>
</li>
<li id="cite_note-10"><span class="mw-cite-backlink"><b><a href="#cite_ref-10">^</a></b></span> <span class="reference-text"><cite id="CITEREFEnganSkrettingHusøy2007" class="citation journal cs1"><a href="Kjersti_Engan" title="Kjersti Engan">Engan, Kjersti</a>; Skretting, Karl; Husøy, John H\a akon (2007-01-01). "Family of Iterative LS-based Dictionary Learning Algorithms, ILS-DLA, for Sparse Signal Representation". <i>Digit. Signal Process</i>. <b>17</b> (1): <span class="nowrap">32–</span>49. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2007DSP....17...32E">2007DSP....17...32E</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.dsp.2006.02.002">10.1016/j.dsp.2006.02.002</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1051-2004">1051-2004</a>.</cite></span>
</li>
<li id="cite_note-11"><span class="mw-cite-backlink"><b><a href="#cite_ref-11">^</a></b></span> <span class="reference-text"><cite id="CITEREFMairalSapiroElad2008" class="citation journal cs1">Mairal, J.; Sapiro, G.; Elad, M. (2008-01-01). "Learning Multiscale Sparse Representations for Image and Video Restoration". <i>Multiscale Modeling & Simulation</i>. <b>7</b> (1): <span class="nowrap">214–</span>241. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.95.6239">10.1.1.95.6239</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1137%2F070697653">10.1137/070697653</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1540-3459">1540-3459</a>.</cite></span>
</li>
<li id="cite_note-12"><span class="mw-cite-backlink"><b><a href="#cite_ref-12">^</a></b></span> <span class="reference-text"><cite id="CITEREFRubinsteinZibulevskyElad2010" class="citation journal cs1">Rubinstein, R.; Zibulevsky, M.; Elad, M. (2010-03-01). "Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation". <i>IEEE Transactions on Signal Processing</i>. <b>58</b> (3): <span class="nowrap">1553–</span>1564. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2010ITSP...58.1553R">2010ITSP...58.1553R</a>. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.183.992">10.1.1.183.992</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FTSP.2009.2036477">10.1109/TSP.2009.2036477</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1053-587X">1053-587X</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:7193037">7193037</a>.</cite></span>
</li>
<li id="cite_note-13"><span class="mw-cite-backlink"><b><a href="#cite_ref-13">^</a></b></span> <span class="reference-text"><cite id="CITEREFMairalBachPonceSapiro2010" class="citation journal cs1">Mairal, Julien; Bach, Francis; Ponce, Jean; Sapiro, Guillermo (2010-03-01). <a rel="nofollow" class="external text" href="http://dl.acm.org/citation.cfm?id=1756006.1756008">"Online Learning for Matrix Factorization and Sparse Coding"</a>. <i>J. Mach. Learn. Res</i>. <b>11</b>: <span class="nowrap">19–</span>60. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/0908.0050">0908.0050</a></span>. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2009arXiv0908.0050M">2009arXiv0908.0050M</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1532-4435">1532-4435</a>.</cite></span>
</li>
<li id="cite_note-14"><span class="mw-cite-backlink"><b><a href="#cite_ref-14">^</a></b></span> <span class="reference-text"><a href="Michal_Aharon" title="Michal Aharon">Aharon, M</a>, M Elad, and A Bruckstein. 2006. "<a rel="nofollow" class="external text" href="https://freddy.cs.technion.ac.il/wp-content/uploads/2017/12/K-SVD-An-Algorithm-for-Designing-Overcomplete.pdf">K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation</a>." Signal Processing, IEEE Transactions on 54 (11): 4311-4322</span>
</li>
<li id="cite_note-15"><span class="mw-cite-backlink"><b><a href="#cite_ref-15">^</a></b></span> <span class="reference-text"><cite id="CITEREFPeyré2008" class="citation journal cs1">Peyré, Gabriel (2008-11-06). <a rel="nofollow" class="external text" href="https://hal.archives-ouvertes.fr/hal-00359747/file/08-JMIV-Peyre-SparseTextures.pdf">"Sparse Modeling of Textures"</a> <span class="cs1-format">(PDF)</span>. <i>Journal of Mathematical Imaging and Vision</i>. <b>34</b> (1): <span class="nowrap">17–</span>31. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2Fs10851-008-0120-3">10.1007/s10851-008-0120-3</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/0924-9907">0924-9907</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:15994546">15994546</a>.</cite></span>
</li>
<li id="cite_note-16"><span class="mw-cite-backlink"><b><a href="#cite_ref-16">^</a></b></span> <span class="reference-text"><cite id="CITEREFRamirezSprechmannSapiro2010" class="citation book cs1">Ramirez, Ignacio; Sprechmann, Pablo; Sapiro, Guillermo (2010-01-01). "Classification and clustering via dictionary learning with structured incoherence and shared features". <a rel="nofollow" class="external text" href="http://www.computer.org/csdl/proceedings/cvpr/2010/6984/00/05539964-abs.html"><i>2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition</i></a>. Los Alamitos, CA, USA: IEEE Computer Society. pp. <span class="nowrap">3501–</span>3508. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FCVPR.2010.5539964">10.1109/CVPR.2010.5539964</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-4244-6984-0</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:206591234">206591234</a>.</cite></span>
</li>
<li id="cite_note-17"><span class="mw-cite-backlink"><b><a href="#cite_ref-17">^</a></b></span> <span class="reference-text"><cite id="CITEREFKoniuszYanMikolajczyk2013" class="citation journal cs1">Koniusz, Piotr; Yan, Fei; Mikolajczyk, Krystian (2013-05-01). "Comparison of mid-level feature coding approaches and pooling strategies in visual concept detection". <i>Computer Vision and Image Understanding</i>. <b>117</b> (5): <span class="nowrap">479–</span>492. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.377.3979">10.1.1.377.3979</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.cviu.2012.10.010">10.1016/j.cviu.2012.10.010</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1077-3142">1077-3142</a>.</cite></span>
</li>
<li id="cite_note-18"><span class="mw-cite-backlink"><b><a href="#cite_ref-18">^</a></b></span> <span class="reference-text"><cite id="CITEREFKoniuszYanGosselinMikolajczyk2017" class="citation journal cs1">Koniusz, Piotr; Yan, Fei; Gosselin, Philippe Henri; Mikolajczyk, Krystian (2017-02-24). <a rel="nofollow" class="external text" href="http://spiral.imperial.ac.uk/bitstream/10044/1/39814/2/pkpami2e-peter.pdf">"Higher-order occurrence pooling for bags-of-words: Visual concept detection"</a> <span class="cs1-format">(PDF)</span>. <i>IEEE Transactions on Pattern Analysis and Machine Intelligence</i>. <b>39</b> (2): <span class="nowrap">313–</span>326. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FTPAMI.2016.2545667">10.1109/TPAMI.2016.2545667</a>. <a href="Hdl_(identifier)" class="mw-redirect" title="Hdl (identifier)">hdl</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://hdl.handle.net/10044%2F1%2F39814">10044/1/39814</a></span>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/0162-8828">0162-8828</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/27019477">27019477</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:10577592">10577592</a>.</cite></span>
</li>
<li id="cite_note-19"><span class="mw-cite-backlink"><b><a href="#cite_ref-19">^</a></b></span> <span class="reference-text"><cite id="CITEREFAlMatouqLalegKiratiNovaraIvana2019" class="citation journal cs1">AlMatouq, Ali; LalegKirati, TaousMeriem; Novara, Carlo; Ivana, Rabbone; Vincent, Tyrone (2019-03-15). <span class="id-lock-subscription" title="Paid subscription required"><a rel="nofollow" class="external text" href="https://ieeexplore.ieee.org/document/8667648">"Sparse Reconstruction of Glucose Fluxes Using Continuous Glucose Monitors"</a></span>. <i>IEEE/ACM Transactions on Computational Biology and Bioinformatics</i>. <b>17</b> (5): <span class="nowrap">1797–</span>1809. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FTCBB.2019.2905198">10.1109/TCBB.2019.2905198</a>. <a href="Hdl_(identifier)" class="mw-redirect" title="Hdl (identifier)">hdl</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://hdl.handle.net/10754%2F655914">10754/655914</a></span>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1545-5963">1545-5963</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/30892232">30892232</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:84185121">84185121</a>.</cite></span>
</li>
</ol></div></div><!--htdig_noindex--><div><div class="zim-footer">
This article is issued from <a class="external text" title="Last edited on 2025-07-24" href="https://en.wikipedia.org/wiki/?title=Sparse_dictionary_learning&oldid=1302204395">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.
</div>
</div><!--/htdig_noindex--></div>
</div>
</main>
</div>
</div>
</div>
</body></html>